Action Point 8 – Strategy to improve access and use of reference standards

During the meeting of WG1 Quality in January 2022, the Member States highlighted the challenges they face in accessing reference standards, including for COVID-19, which is critical to test the quality of the products on the market. These concerns were driven by the lack of existing reference standards for some products (e.g. COVID-19 vaccines), but also difficulties in accessing reference standards due to insufficient information about possible sources, challenges in producing reference standards, and the high costs associated with procurement and supply (including transport).

To address this need, the Assembly adopted on 8 June 2022 the SEARN Work Plan 2022-2023 which includes the action point 8, led by Working Group 1 (WG1) Quality: ‘Draft a strategy to facilitate the identification needed and sharing of reference material/chemical reference standard in the region’.

As per the WHO General guidelines for the establishment, maintenance and distribution of chemical reference substances, a chemical reference substance is an ‘authenticated, uniform material that is intended for use in specified chemical and physical tests, in which its properties are compared with those of the product under examination, and which possesses a degree of purity adequate for its intended use’ (1). The same guideline distinguishes:

primary chemical reference substance, widely acknowledged to have the appropriate qualities within a specified context, and whose assigned content when used as an assay standard is accepted without requiring comparison with another chemical substance.
secondary chemical reference substance, a substance whose characteristics are assigned and/or calibrated by comparison with a primary chemical reference substance.
International Chemical Reference Substance, primary chemical reference substances established on the advice of the WHO Expert Committee on Specifications for Pharmaceutical Preparations and supplied primarily for use in physical and chemical tests and assays described in the specifications for quality control of drugs published in The International Pharmacopoeia or proposed in draft monographs.
Pharmacopeial reference standards, established and distributed by pharmacopeial authorities following the general principles of the ISO Guide: General requirements for the competence of reference material producers (ISO 17034:2016).

Further, the WHO Recommendations for the preparation, characterization and establishment of international and other biological reference standards also identify biological reference standards which are critical for biological medicines and in vitro diagnostics (2). Biological reference standards usually comprise materials of complex composition that require biological or immunological assay for appropriate characterization, and are intended to facilitate standardized characterization of biological samples, whatever the type of measurement or method used.

Reference standards for IVDs typically focus on in vitro measurement procedures used for diagnosis, detection and management of infectious diseases where the typical analytes (measurands) are nucleic acid (NA) or antigen (Ag) or antibodies (Ab). Due to their complexity (that is, the epitope spectrum represented by polyclonal antibodies in the serum/plasma), antibody-based secondary standards should be approached with caution.

International reference standards (highest order, international conventional calibrator) such as WHO International Standard (IS). They are solely established by the WHO Expert Committee on Biological Standardization. Establishment of a WHO IS follows a collaborative study involving various users of the material (including national control laboratories, IVD manufacturers and other accredited laboratories) and as many different, well-established assays as feasible. The material used should resemble as closely as possible the natural analyte of the clinical sample to be measured. An assessment of commutability should be performed as part of the collaborative study where appropriate and feasible. By definition, an IS has a specified value expressed in International Units (IU). This value is arbitrarily assigned based on the results of the collaborative study. WHO IS should not be used for more routine procedures such as validation of assays and as run controls.

Secondary standards (Regional or national reference materials, laboratory or manufacturer’s working calibrator), which is calibrated against the IS. The titre, composition and method of production of secondary standards will vary but should be suitable for obtaining sufficient measurements, when dilution is needed, to achieve an accurate calibration. Regardless of the method of production, each calibration will have a stated measurement uncertainty.

Tertiary standards (external control materials, working reagents or standards, manufacturer’s product calibrator), which is calibrated against the secondary standards. The standard may be formulated from either biological (for example, patient-derived) or non-biological material. However, regardless of the material used, all references in the traceability chain should also demonstrate commutability to the clinical sample of the tested analyte. Tertiary standards are typically formulated as a liquid preparation and may comprise a concentration of the analyte that is detected without dilution in the linear range of the assay it is intended for. They will often be used as an external control material in addition to that normally supplied by the assay manufacturer. Regular monitoring of such material may allow for the early detection of problems with assay performance.

The behaviour of the reference standard should resemble as closely as possible the behaviour of test samples in the assay systems used to test them. As per WHO, ‘The concept of commutability seeks to establish the extent to which the reference standard is suitable to serve as a standard for the variety of samples being assayed’. (TRS 943 - Annex 3: WHO general guidelines for the establishment, maintenance and distribution of chemical reference substances).

The following possible sources of Reference Standards Sources for SEARN QCLs were identified:

Name	WHO Region	Country
Primary Reference Standard
USP	PAHO	US	Chemical
BP	EURO	UK	Chemical
EP	EURO	France	Chemical
EDQM/WHO	EURO	France	Chemical
IPC	SEARO	India	Chemical
Secondary Reference Standard
ASEAN	SEARO	Thailand	Chemical
INCQS	AMRO	Brazil	Chemical
DMSc	SEARO	Thailand	Chemical
NQCLDF	SEARO	Indonesia	Chemical
Further links are provided on the WHO website to webpages of pharmacopoeias that provide information on reference substances established to support the performance of analytical provisions described in the respective pharmacopoeias: https://www.who.int/teams/health-product-policy-and-standards/standards-and-specifications/norms-and-standards-for-pharmaceuticals/international-pharmacopoeia/IMWP

Supplier selection

-thorough review of suppliers by evaluating credentials, QMS, and experience

-shortlist suppliers and invite proposals

Evaluate proposals

-evaluate proposals against criteria such as price, purity, content, delivery time, and customer service.

-identify top suppliers and request samples

Sample testing

-full sample testing of the reference standard to verify its purity and content

-on-site assessment to verify QMS, processes, and operations

-evaluate ability of source to meet requirements in terms of purity, content, and reliability

-we could use “RELIANCE” and rely of assessment reports from other respected bodies

Evaluation report

-prepare evaluation report summarizing results of the evaluation of sources and assesses suitability to meet requirements

-use evaluation report to make final decision

On-going monitoring

-once source is selected, establish a process of on-going monitoring of performance to ensure reference standards continue to meet the requirements

-schedule regular assessments and sample testing

-could also use “RELIANCE” in the monitoring of performance of source

Supplier selection

-preliminary review of suppliers based on their credentials and experience in supplying reference standards

-shortlist suppliers who meet minimum requirements and request quotes

Evaluation of Quotes

-evaluate quotes against criteria such as price, purity, content, and delivery time

-identify the top suppliers that offer the best value for money and request samples

Sample testing

-choose a limited number of tests which are critical to give a good representation of the reference standard’s quality

-conduct these tests on reference standards samples to verify their purity and content

Supplier communication

-contact shortlisted suppliers to discuss their products and services

-clarify any details or specifications that may affect reference standard’s quality

Evaluation report

-prepare a report summarizing the results of the evaluation and assesses suitability of each supplier to meet requirements

-use evaluation report to make final decision

On-going monitoring

-establish a process of monitoring performance of supplier(s)

Needs Assessment and Alignment: Conduct a needs assessment of the regional NQCLs to identify the specific reference standards required. This survey should be held on a periodic basis, such as annual, biannual or trienal.
Supplier Identification and Evaluation: Identify an established supplier, with a comprehensive range of reference standards (primary, secondary, working, certified reference material). Evaluate based on their reputation, product quality, compliance with international standards, and ability to meet the regions’ requirements, ensuring alignment between the identified needs and capabilities of the potential supplier(s), and qualify the supplier.
Contract/MoU Negotiation: Initiate discussions with the chosen reference standards supplier to negotiate the terms of collaboration/contractual terms, keeping in mind the type of standards, the expected number of necessary units (to be able to determine a minimum stock per standard), special transport conditions, expiry dates, among other items.
Documentation and Legal Framework: Draft a comprehensive Contract/MoU that outlines the terms and conditions of the collaboration. Important clauses to include, intellectual property rights, confidentiality, liability, dispute resolution, and termination procedures.
Infrastructure and integration: Establish necessary infrastructure to integrate the reference standards from supplier into the regional network of NQCLs, such as a centralized database/digital platform to manage inventory. An additional option could be to get the supplier to do this as well.
Logistics and Distribution: ensure that logistics and distribution protocols, with suitable transport validation, are implemented by the supplier, to ensure timely and efficient delivery from supplier to NQCLs.
Quality Assurance and Monitoring: Implement a QA program to verify integrity and compliance of the reference standards received from supplier.
Review and Continuous improvement: Regularly review the effectiveness of the collaboration and the satisfaction of the NQCLs with reference standards provided, namely thru supplier requalification. Seek feedback from NQCLs to identify areas of improvement and make necessary adjustments to the collaboration.

Needs Assessment and Alignment: : Conduct comprehensive needs assessment to determine specific requirements and standards needed by NQCLs in the region. This survey should be held on a periodic basis, such as annual, biannual or trienal.
Resource planning: Allocate resources for establishing the CRRS, namely financial, structural and human resources.
Supplier Identification: Research and identify potential suppliers with a comprehensive range of reference standards (primary, secondary, working, certified reference material).
Supplier evaluation: Evaluate the supplier(s) based on their reputation, product quality, compliance with international standards, and ability to meet the regions’ requirements, ensuring alignment between the identified needs and capabilities of the potential supplier(s), and qualify the supplier(s).
Technical Agreement/MoU Negotiation: Develop a comprehensive technical agreement/MoU outlining terms of collaboration with the chosen reference standards supplier(s). Include aspects such as supply agreements, pricing, quality control, delivery schedules, return policies and dispute resolution mechanisms.
Infrastructure Setup: Establish the necessary infrastructure for the CRRS, including storage facilities, equipment for handling and cataloging reference standards, an inventory management system to track, stock/availability and expiry dates of reference standards, as well as human resources needed to manage the CRRS.
Procurement and Inventory Management: Procure the required standards based on needs assessment and update the inventory.
Training and Documentation: Provide training to staff members responsible for managing the CRRS on proper storage, handling, and documentation procedures.

Procuring reference standards may be difficult in certain areas of the world due to delays in their delivery and the cost of purchase. The number of reference substances prescribed in authoritative analytical procedures shall therefore be reduced, if possible.

Acknowledging this situation, The International Pharmacopoeia applies, for example, the following strategies and practices when elaborating monographs:

in situ preparation of impurities for identification of related substances/impurities;
establishment of compounded reference substances that contain several potential impurities for identification;
quantification of impurities by comparing their detector responses with the response of the parent compound in a diluted sample solution along with the establishment of correction factors to compensate for differences in the responses of the impurity and the parent compound;
provision of International Infrared Reference Spectra (IIRS) for use in identification tests;
provision of assay methods not requiring reference substances, like titrations and ultraviolet spectrophotometry using absorptivity values. These methods shall be provided as alternatives in particular to chromatographic assays in monographs for pharmaceutical substances.

These strategies are only be applied when, during the elaboration of the methods, evidence has been obtained demonstrating that the intended measures do not compromise the quality of the analytical results or the ability of the tests to conclusively demonstrate conformance to the applicable standards.

Secondary reference substances shall be traceable to primary reference substances. Primary reference substances established by WHO or pharmacopoeial authorities represent the highest order reference standard. Other lower-order reference standards for a given analyte can then be related through a sequence of comparisons traceable to the primary standard. Therefore, secondary standards, calibrated directly against the IS, should be established for use in the calibration of tertiary or working standards. Regional or national reference materials are usually secondary standards. In addition, manufacturers and research laboratories performing large numbers of assays may develop secondary standards calibrated directly against the primary standard. When developing and assigning a value to any lower-order reference standard, evaluating the likely impact of accumulated uncertainty, and determining whether this is acceptable given the intended use of the standard, will be an important element in the study design. If need be, the uncertainty of the assigned property has to be reduced by increasing the number of laboratories participating in the collaborative trial.

In case reference standards are not commercially available, the opportunity to obtain them from manufacturers shall be explored. The user will have to verify the documentation accompanying the reference standard and, where insufficient information is provided, confirm the suitability of the reference standard by undertaking, where possible, appropriate analytical investigations, the extent of which depends on the intended use(s) and is based on scientific judgement.

If no reference substances can be made available, the establishment of in-house reference substances should be explored. Candidate material may be selected from the normal pharmaceutical production provided that the purity and homogeneity are suitable. The candidate material should be tested with suitable analytical techniques aiming to characterize all relevant quality attributes. A thorough purity investigation is performed to verify the presence of all relevant components (i.e. main component, organic and inorganic impurities, water and residual solvents) and to quantify them, if relevant. The cumulative percentage of all components of standards used for quantitative purposes should yield 100% (mass balance approach). The purity of a candidate material is calculated on the “as is” basis, so that the analyst can use the reference substance without pretreatment. If in-house reference substances are used as secondary reference substances their assigned contents should be calibrated against official primary standards.

The use of reference standards for a scope outside the intended use(s) should be avoided. If this is inevitable, it is the user´s responsibility to demonstrate the suitability of the reference substance for the new intended use(s), usually be performing supplementary tests. Exceptionally, confirmation may be also be obtained by scientific justification/rationale. In cases of doubtful results or dispute, however, tests performed using authenticated reference substances as per their intended purposes are the only authoritative ones.

The implementation of reliance as a general principle to foster collaboration and to make best use of available resources and expertise will also help national quality control laboratories to rationalise the use of reference substances. Reliance between laboratories will be supported by activities such as:

Joint establishment of reference substances;
Common procurement of candidate material for reference substances;
Harmonizing procedures for the establishment of reference substances;
Setting up of working groups with a view to exchange information, to discuss challenges and to seek common solutions for problems associated with the establishment of reference substances
Participation in collaborative trials to establish reference substances (WHO organizes collaborative trials to establish International Chemical Reference Substances. Laboratories wishing to participate in these trials may contact [email protected]).
Evaluation Of Reference Standards Sources/Suppliers

Further opportunity for worksharing include:

Inter-laboratory testing of identity, purity and assay of reference standards may be performed within the regional NCLs to assign potency of the reference standard.
Selected List of vendors for raw materials to be used to prepare reference standard and test result of such raw materials for identity, purity and assay may be shared with other labs.
Test Procedure developed by an NCL for testing of candidate raw materials intended to be used for preparing reference standard may be shared with other NCLs.
Training on testing of reference standard may be undertaken by an NCL for other NCL staff.
Calibration of secondary reference standard against primary standard may be shared among the NCLs in the region.

Finally, cooperation between the NCLs and information sharing through the use of emails or the forum on the SEARN internal platform could be developed to facilitate sharing and testing of samples in the NCLs which have the capacity and the required reference standards.

General guidelines for the establishment, maintenance and distribution of chemical reference substances - TRS 943 - Annex 3 [Internet]. 2007. Available from: https://www.who.int/docs/default-source/medicines/norms-and-standards/guidelines/quality-control/trs943-annex3-establishmentmaintenance-distribution-chemica-reference-substances.pdf?sfvrsn=71064286_0
Recommendations for the preparation, characterization and establishment of international and other biological reference standards, Annex 2, TRS No 932 [Internet]. 2004. Available from: https://www.who.int/publications/m/item/annex2-trs932
WHO manual for the preparation of secondary reference materials for in vitro diagnostic assays designed for infectious disease nucleic acid or antigen detection: calibration to WHO International Standards, Annex 6, TRS No 1004. https://cdn.who.int/media/docs/default-source/biologicals/blood-products/document- migration/secstandmanwho_trs_1004_web_annex_6.pdf?sfvrsn=d6937af8_3&download=true

Action Point 8 - Strategy to improve access and use of reference standards

Background

Scope and definitions

General

Reference standards for IVDs

Objectives

Reliable sources of reference standards

Reliable sources of Chemical Reference Standards

Evaluation Of Reference Standards Sources/Suppliers

Plan 1 (comprehensive)

Plan 2 (economical)

Improving access to Reference Standards

Approach 1: Establishing specific procurement contracts with an established and duly qualified Reference Standards Supplier

Approach 2: Establishing a Central Repository for Reference Standards (CRRS)

Alternatives to reference standards

Reducing the use of reference standards

Principles for establishing and using secondary reference substances

Addressing situations when no affordable reference standards is available

Optimizing the use of reference standards through reliance, worksharing, and collaboration

Next steps

References